Archiving Data

SUMMARY

It is described how accrued data on process visualization can be archived.

CONTENTS

  1. Introduction
  2. Archives in Alpha-Vision
  3. Creating a long term archive
  4. When will it be necessary to save the long term archive.
  5. Storing of ARCHIVES

Introduction

When starting the process visualization data are changed by the process flow and by controlling of the process. The process pictures reflect these changes.
System events are logged automatically in the "_EVENTLOG.DBF" archive.
Value changes not initiating an event may also be logged. These data are saved to a long term archive named "_ARCHIVE.DBF" but the procedure to do so must be set up manually.

For all archives in ALPHA-VISION® applies:

Should the process flow be documented for a longer period of time-, or be analyzed later on, the actual archives must be saved before data are overwritten.

Archives in Alpha-Vision

Eventlog
System events are stored in the "_EVENTLOG.dbf"; it is written automatically from system start on. The size for that archive is defined in the <Project name>.cfg file.

Archives for data logging
For data logging we distinguish between short term- and long term archives.

SHORT TERM ARCHIVE
The short term archive is mentioned for the sake of completeness only. It can’t be saved and is typically used on graphs in process pictures.
See: graphs

LONG TERM ARCHIVE
The long term archive got always the name "_ARCHIVE.dbf" and is created in runtime. In the archive, measuring data of an unlimited number of Data-Objects can be stored; it may have a size of up to 1 GB.
If the long term archive should be written it must be configured in advance.

For the long term archive applies:

DcLog
In the "_DCLOG.dbf" file system events of the Damage Control System are stored.

Note:
The DC-System is an ALPHA-VISION® add-on software package.

Creating a long term archive

If values for ALL Data-Objects should be recorded the archive parameters can be entirely defined in the Vision2000.ini file, Section "DoArch".

If only selected Data-Objects should appear in the archive it can be done via the program DoArch of the PMU; in that case the project has to be compiled.

When will it be necessary to save the long term archive.

Basic approach for that consideration will always be the period of time data should be stored for.
It has to be calculated whether all data records to be stored for that period can safely be written to the archive before it is overwritten – if not, data must be stored in between as often as necessary.
Since only value changes are recorded the frequency of change is the vital factor for the period of time that can be stored in an archive with predefined size.

Parameters in the Vision2000.ini file, section "DoArch" that have influence:
Parameter Meaning
SampleInt: Sample Interval, the interval the value is checked for a change; changes in between the intervals are not recorded.
Threshold: Only value changes exceeding the defined value are recorded.
NumArchRecords: Number of records that can be stored.

Example of a calculation of the expected data volume
The value change of 500 Data-Objects should be recorded.
For 50 Data-Objects a change frequency of 20/min is expected = 1000/min
For 150 Data-Objects a change frequency of 5/min is expected = 750/min
For 300 Data-Objects a change frequency of 0,1/min is expected = 30/min

For safety reasons 2000 changes per minute are taken into account.

The entry “NumArchRecords“ in section "DoArch" of the Vision2000.ini file is set to 150000 – the number of value changes that can be stored.
(Any data record got 67 bytes – the file header got 161 bytes; maximum file size is 1 GB.)

Assuming 2000 changes per minute the archive is full after 75 minutes and the oldest entry is overwritten by a new one.

In that constellation data for the last 75 minutes can be provided.
If that is not sufficient the file "_ARCHIVE.dbf" must be copied somewhere else.

Storing of ARCHIVES

If it is necessary to save the "_ARCHIVE.dbf" the according parameters in the Vision2000.ini file, section [ZipArchives] have to be defined and the program AvZipArchives.exe must be started in the runtime.

See also ZipArchives-Systematics and Configuration

Parameters in the vision2000.ini that must be defined
WriteInterval
Defines the interval in hours after that data are copied from the "_ARCHIVE.dbf".
Time intervals start at 00:00 h, possible values are 1,2,4,6,8,12,24.
Example:
If the runtime starts at 13:00 with the WriteInterval set to 4, the first time a WriteInterval ends is 16:00.(00-04-08-12-16)

CheckTimeInterval
Defines the interval in minutes the Write Interval is checked for expiration.
Possible values are 1 to 30 minutes.

Dbf_<n>
Defines the files to be stored. (1=_archive, 2=_eventlog, 3=_dclog)
Note:
Should the stored data be analyzed by "AvReplay" the files "_Archive.dbf" AND "_Eventlog.dbf" must be saved.
In that case the expected amount of data for the Eventlog has to be calculated too; the higher value is determining for the storage cycle.

Path_<n>
Defines the path for the files to be saved.

Parameters that can be defined
ZipDrv
Defines where the saved files should be stored. In the defined folder a subfolder "_Archive" for the compressed daily-archives <YYYYMMDD.zip> is created
If the "_Eventlog.dbf" is earmarked for storing, a subfolder "_Eventlog" for the compressed eventlogs <YYYYMMDD.zip> is created.

Note:
If the parameter is not defined, the subfolder "_Archive", ("_Eventlog") is created in the same folder where the program "AvZipArchives.exe" is located.

WARNING
If there is no space for the new daily-archive, the oldest one is overwritten.

"DestDrv"
If the defined path is different to that one defined below "ZipDrv" the files are copied from there to:
<DestDrv>\_Archive\ and, if existent
<DestDrv>\_Eventlog\


Schematical sequence on storing archive data

Assumptions and Definitions:

The first storing of the daily-archive of the 2009/11/11 is initiated at 08:03; it is the first “CheckTimeInterval” after expiration of the first “WriteInterval”.
Since there is no definition where to put the file, it is written as "1111_08.dbf" to C:\Vision\bin\_Archive. Thereby only data records from 00:00h to 08:00h are copied out of the "_ARCHIVE.dbf". Immediately afterwards the file is compressed to the daily-archive "20091111.zip" in the same folder and the file "1111_08.dbf" is deleted.
The same procedure is initiated for the "_Eventlog.dbf".

The second storing for the daily-archive of the 2009/11/11 is initiated at 16:06, the first “CheckTimeInterval” after expiration of the second “WriteInterval”. It is written as "1111_16.dbf" to C:\Vision\bin\_Archive. Thereby only data records from 08:00h to 16:00h are copied out of the "_ARCHIVE.dbf". Immediately afterwards the file is compressed and added to the existing daily-archive "20091111.zip" in the same folder, the file "1111_16.dbf" is deleted.
The same procedure is initiated for the "_Eventlog.dbf".

The third storing completes the daily-archive of the 2009/11/11. For the 2009/11/12 a new daily-archive named "20091112.zip".